Apparatus and method for finding a misplaced object using a database and instructions generated by a portable device

ABSTRACT

The basic invention uses a portable device that can contain a camera, a database, and a text, voice or visual entry to control the storage of an image and its location into a database. The database can be distributed over several memory arrays in different devices. Furthermore, the stored image can be associated with text, color, visual or audio data. The stored images can be used to guide the user towards a target if the user does not recall the current location. The user&#39;s commands can be issued verbally, textually or by scrolling through the target images in the database until the desired one is found. This target can be shoes, pink sneakers, a toy or some comparable items, etc. that the user desires to find.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to the co-filed U.S. applicationentitled “APPARATUS AND METHOD FOR IDENTIFYING A NAME CORRESPONDING TO AFACE OR VOICE USING A DATABASE” filed on Nov. 14, 2006, which isinvented by the same inventor as the present application andincorporated herein by reference in their entireties.

BACKGROUND OF THE INVENTION

Failing to remember where one placed an item can be frustrating. Manytimes, the remote control of a TV can't be found, finding a pair ofmissing pink sneakers causes one to miss catching the school bus ontime, or losing a loved toy that requires love and huggies is a sadevent.

The current method of locating these items is by an exhaustive physicalsearch. As “Mom” is telling you to hurry up, the bus is coming, “wheredid you put your sneakers?” You get even more frantic. The search spaceyou are viewing may in fact contain the sneakers but due to the rapidityand shortness of time, your brain fails to recognize it. Then you missyour bus. Now, your “Mom” has to drive you to school.

Several previous inventions address the problem of locating a target.One, U.S. Pat. No. 7,113,089, issued Sep. 26, 2006 describes a portabledevice that can be used to identify a target using wireless signals. Thetarget and the portable device must have a wireless transponder insideof them. The user enters a desired target into the handheld and movesthe handheld to locate the best signal strength and moves in thatdirection. Two weakness of this invention are 1) the target must containa wireless transponder and 2) the method of locating the target uses amaximum wireless signal strength which can be inaccurate sincereflections from walls may mislead the search for the target.

A second reference, U.S. Pub. No. 2006/0009167, filed Apr. 7, 2005describes a technique for guiding a person along a path in an adverseenvironment such as a fire, chemical spill, etc. The user wears aprotective suit and drops wireless transponders along the path they aremoving along. The transponders are dropped from a backpack automaticallyat regular intervals and the user can use these transponders to find hisway back. This is equivalent to the procedure that “Hansel and Gretel”used by dropping bread to find their way back from the woods.Unfortunately for “Hansel and Gretel”, the birds ate the bread so theycouldn't return. But in the second reference a wireless transponder isused not to locate a target but provide for a way of guiding a useralong a path to get back to the beginning. As stated previously, thesecond reference does not offer a way of finding a target.

Visual target recognition is a matching algorithm or recognition unitthat can compare a target with a stored version in the database in anattempt to get a match. Aircraft and missiles used visual guidancesystems can use a visual display to find a target as described in U.S.Pat. No. 6,349,898, issued Feb. 26, 2002. The visual system along withthe recognition unit can be combined and used to guide the missile tothe target. The target recognition system inside the missile is used tocontrol the and direct the missile to its target for ultimatedestruction. However, the user needs to see the target before releasingthe missile. If the target's location is already known before therelease of the missile, then the target is not lost. This is in contrastto the description of the invention in this application where thetarget's location is not known to the user.

It is desirable to have a tool that can look for and find lost items sothat life can continue at its normal pace. Since many lost items do notcontain an RF tag or wireless transponder a different way of locatingthese lost items is required. It is the purpose of this invention tofind a target that does not have wireless tags or any type of wirelessdevices embedded in the desired or possible lost target.

BRIEF SUMMARY OF THE INVENTION

The invention described in this application is a portable device that isused to locate a target that has been lost or misplaced without forminga wireless link between the portable device and the misplaced target.Thus, the target does not necessarily contain a wireless tag or wirelesstransceiver embedded in the lost target. This target is an object thattypically can easy to move (mobile); such as, a remote control for a TV,a small toy, headgear, shoes, sneakers, school books, back pack orhomework papers. Mobile in this case implies that the target can bepicked up, used, moved, and set down in a new location in a very shortperiod. Typically, the new location where the target is placed isquickly forgotten and the user can not locate the target when theyrequire it at a later date. The invention stores the association of thecurrent location along with the target and places this association intoa database.

Immobile objects can be recognized and used to provide key positionanchor locations. These immobile objects can create a framework withinthe database to help navigate the user to its target. The next level ofimmobile objects such as a table, refrigerator, couch, etc. are lessimmobile but can serve as a secondary level of anchor framework to alsohelp move the user through the environment to help location the targetobject. The locations of the very mobile objects are compared to the twolevels of immobile objects and a relative position is recoded in thedatabase.

The target may have had already existed in the database if it was storedat an earlier date. The user can locate the target in the portabledevice by using voice activation or textual commands. In addition, avisual image can be scrolled on the display screen of the portabledevice to identify a previously stored target. If the portable devicedoes not recognize the new target in the database of the portabledevice, then the user issues a prompt to store the new target into thedatabase.

Once a desired target is required, and the user can not locate thetarget, then the user uses the portable device in an attempt to identifythe target's location. The portable device determines the currentlocation of the user after the user desires to locate the target. Theportable device analyzes its database to locate the target and directsthe user to move in a given direction as specified on the display screenof the portable device. The portable device continuously updates thecurrent location of the moving user as the target is being approached.Once the target is located, it is used for the particular function thatthe target offers, i.e. put sneakers on feet.

In some cases, the target may have been moved by another user in thehouse. This second user also uses their portable device. Thus, sincetarget has been moved by a different user and not the original user, thelocation of the target is contained within this different user'shandheld database. Since the database can exist within the portabledevice, in a PC, in a server, or the database that is contained inanother user's portable device; the first user's portable device canquery these databases including the different user's database and updatethe first user's database. There will have to be timestamps on all itemsthat are stored into the database to insure that the newest location isused.

The portable device can also use voice, textual or images presented tothe user by the portable device to locate a misplaced target. One imageis an arrow pointing in the correct direct, while another is a visualdisplay of the current location highlighting a door which should beentered. A third method is coloring the display screen with shades ofcolor; blue for cold, yellow for warm and red for hot. The portabledevice can also issue verbal commands to direct the user or textualinstructions displayed on a display screen of the portable device.

This invention has several advantages; 1) the target does not require awireless transponders; thus if the battery of the transponder lossespower, this new invention can still find the object, 2) as one target isbeing searched, other targets in the database can be located andconfirmed to be at the known location, 3) if a target can not be locatedin the expected location (i.e., a different user moved it), the databaseof other users can be searched to potentially located it; thus, thetarget can be located even if the target is moved by a different user.This is particularly advantageous if other people reside in the sameresidence and move the target, 4) the portable device only requires asearch and compare algorithm to locate the target, 5) several differentinstructional means (textual, visual, colored screen, or audio) can bespecified to emanate from the portable device to help give instructionsto locate the target.

With the prevalence of portable devices and the increase incomputational ability of many portable devices, the ability of findinglost targets can be implemented into a portable device that contains alocal database.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 a illustrates a prior art apparatus displaying an image on thedisplay screen of a portable device.

FIG. 1 b depicts a prior art apparatus displaying a second image on thedisplay screen of a portable device after rotation.

FIG. 2 a shows a user carrying a portable device that collects imagesand stores them into memory made in accordance with the presentinvention.

FIG. 2 b illustrates the user moving across the room and collecting andstoring addition images of items along the path made in accordance withpresent invention.

FIG. 3 a depicts the user attempting to use the portable device tolocate an object in accordance with the present invention.

FIG. 3 b shows the user being directed to the object via the displayscreen in accordance with the present invention.

FIG. 4 illustrates a flowchart indicating the scanning procedure inaccordance with the present invention.

FIG. 5 depicts a second flowchart giving the procedure to location anobject in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 a provides a prior art view 1-1 a of a user 1-2 viewing aportable device 1-3 to display an image 1-9 a on the display screen 1-4.The portable device 1-3 also contains at least one antenna 1-5 and adatabase (not shown). The wireless signal 1-11 can be used to connect toa second external database to obtain additional information. Theportable device 1-3 also contains a camera 1-6. This portable device 1-3can be a cell phone, a PDA device, or any device that contains a displaycomponent, antenna and a camera. The display screen 1-4 can be an LCD,TFT, or any comparable display.

An image propagation path 1-7 a of a first target 1-8 a is sensed by thecamera 1-6. The image 1-9 a is displayed in accordance with the presentinvention played on the display screen 1-4 to the user 1-2. The capturedimage propagation path 1-10 is sensed by the user 1-2.

Although the targets are illustrated using faces, these targets can bean item in the vicinity of the user; for instance, window, box, shoe,doll, table, door, etc.

All of the components identified with the same numeric throughout thesedrawings corresponds effectively to the same component or item. Forexample, in FIG. 1 b another prior art drawing 1-1 b is illustrated. Theportable device 1-3 is rotated 1-12 to capture the image of a secondtarget 1-8 b. An image propagation path 1-7 b of a second target 1-8 bis sensed by the camera 1-6. The image 1-9 b is displayed on the displayscreen 1-4 to the user 1-2. The captured image propagation path 1-10 issensed by the user 1-2.

FIG. 2 a shows a portion of a hallway 2-1 a. One wall contains a firstwindow 2-3, a door 2-5 and a second window 2-9. Note that these areimmovable targets in the sense that they would require major renovationsto the hallway to move these targets into different locations. Forinstance, a contractor would be called into reposition the window 2-9three feet lower. Thus, some targets are more permanent to a firstdegree than other targets.

In FIG. 2 a, two targets 2-7 and 2-10 are also in the hallway 2-1 a.These targets are easier to physically move from their initial location.The user 1-2 moves the portable device 1-3 along the hallway 2-1 arecording the images of the various targets; both immovable andmoveable. The software embedded in the portable device is intelligentand can make a decision that a window or door is more permanent and showbe used to navigate the user through the hallway at a later date.

In addition, as each target is captured as an image, the user identifiesthe target. For instance, the image 2-4 is identified by the user 1-2 asa window, the image 2-6 is identified as a door, and image 2-7 isidentified as doll. Each identified item can be recalled on the portabledevice at a later date using text, voice or viewing a set of images onthe portable device's screen.

The portable device collects the image 2-4 of the window 2-3. The image2-6 of the door 2-5 is stored and the image 2-8 of the first target 2-7is captured. Note that the portable device is in motion 2-2 with theuser 1-2.

In FIG. 2 b, the user 1-2 is now recording the image 2-11 of the window2-11 and the image 2-12 of the second target 2-10. In this case, theuser 1-2 does not identify the target as a pink sneaker. Only the imageof the pink sneaker is stored in memory. The user is shown to be inmotion 1-4, although this action may not always be required.

In FIG. 3 a, the user enters the desire to locate the pink sneaker intothe portable device via text or voice. Since the association of thesneaker with either test or voice has never been made currently or at anearlier date, the user 1-2 then scans the images in the database for theimage of the sneaker and finds the pink sneaker. The user 1-2 requeststhe portable device 1-3 to provide directions to the user 1-2. Thecommand can be a button on the portable device that is depressed whilethe display unit shows the sneaker or a voice command can be used tostart the search.

The portable device 1-3 illustrating the direction the user 1-2 shouldmove. These directions can be an arrow 3-3 a displayed on the displayscreen. The user 1-2 follows the arrow 3-3 a. The arrow 3-3 a directsthe user to move 3-2 b along the hallway 3-1 a. The portable deviceviews the environment and compares it to the database that was generatedat an earlier date.

FIG. 3 b shows the user 1-2 viewing the screen and monitoring the arrow3-3 b indicating that the target 2-10 is the item of interest. Aspointed out earlier, the target 2-10 was equivalent to the sneaker.Thus, the user 1-2 can now quickly get dressed and make it to school ontime. Otherwise, without the use of the invention, she would be late forthe bus and miss a day of school.

A flowchart 4-1 is depicted in FIG. 4 that describes the scanningprocess. Moving from the block 4-2 to the block 4-3 causes a new imageto be scanned into the portable device. Block 4-4 determines if this isthe first scan. If not, then block 4-5 connects the current image withthe previous image. Otherwise, block 4-6 indicates that the image isviewed on a display screen. Then, 4-7 indicates that the entire image isstored into a database. A decision block 4-8 determines if an object isidentified in the image. If so, block 4-9 places boundaries around theobject in the stored images. These component images that have beenidentified are associated with windows, doors, shoes, etc. Then, block4-10 compares the object with the database and finds a match. Thedecision blocks 4-12 checks if the position of the object has beenidentified. If not, then move the block 4-3; otherwise, the block 4-13is entered and the scanning is complete.

A second flowchart 5-1 is illustrated in FIG. 5. Block 5-2 enters theobject into the portable device. The next block 5-3 searches thedatabase for this object. The search (not shown) can be textual, verbalor visual. A decision block 5-4 determines if the object is in thedatabase. If not alter the search parameter or select a new object asblock 5-5 indicates.

Otherwise, enter block 5-6 to determine the user's current position byscanning the surrounding area and matching the images to the stored scanmade at an earlier time. Then, block 5-7 determines the absoluteposition of the desired object within the database. In block 5-8, analgorithm then maps a path from the current location to the absoluteposition. Control moves to block 5-9 where directions are issued to theuser to move along the path.

A decision block 5-10 determines if the current relative position haschanged significantly. If so, return to block 5-6 to improve theestimate of the relative position. Otherwise. follow the directions asgiven in block 5-11. If object has not been found, as indicated in thedecision block 5-12, return to block 5-6 to achieve a better estimate.Otherwise, the directions have located the object as indicated in block5-13 and the process of locating the object is complete.

It is understood that the above descriptions are only illustrative ofthe principles of the current invention. In accordance with theseprinciples, those skilled in the art may devise numerous modificationswithout departing from the spirit and scope of the invention. Inaddition, the portable device can always be searching the environmentwhile the user is moving around their environment in a non-search mode;this allows a continuous update to the very mobile object in casesomeone else has moved these objects. Thus, the database will always beupdated as the portable device is moved from one location to the next.The portable device can be embedded into a small form factor (i.e.,remove the display screen, for example) and placed in a pair of glassesor into the ear canal. The commands in this later case would be audioand the unit can then always view the environment and continuouslyupdate the database. The portable device can be a flip-phone, an iPod,Blackberry or any wireless handheld unit. Finally, the database can befully contained in the portable device or may exist though a wirelessaccess such as Bluetooth, 802.11, UWB, or a similar wireless protocol inanother portable device, a personal computer or a server.

1. An portable device apparatus comprising; at least one camera thatscans a region of space producing at least one image; the device ismoved to scan an adjacent region of space to produce additional images;all of the images are stored in a database; the images within thedatabase are pasted together forming a composite image; whereby arecognition unit is used to identify an object within the compositeimage.
 2. The apparatus of claim 1, wherein a location of the identifiedobject can be determined.
 3. The apparatus of claim 2, wherein a searchengine can be used to locate the identified object within the database;and issue instructions from the portable device to move the devicetowards the identified object.
 4. The apparatus of claim 1, wherein thedatabase can be distributed over different physically distinct memories;wherein each memory can be individually queried.
 5. The apparatus ofclaim 1, further comprising an instruction means using the portable unitand the recognition unit to identify the identified object.
 6. Theapparatus of claim 5, wherein the instructions includes an audiblesignal to direct the user towards the identified object.
 7. Theapparatus of claim 5, wherein the instructions includes an arrowpointing towards the identified object.
 8. The apparatus of claim 1,further comprising a means for continuously scanning the regions ofspace.
 9. The apparatus of claim 8, wherein the location of theidentified objects are updated in the database if these objects aremoved.
 10. A method of identifying the position of an object in adatabase comprising the steps of; scanning a new image using a camera ina portable device; viewing portions of the image on a display screen ofthe portable device; storing the entire image into a database;identifying at least one object within the entire image; placingapproximate boundaries around the object in the image; comparing theobject against the objects in the database; and finding a match; therebyidentifying the position of the object in the database.
 11. The methodof claim 10, wherein the match is determined by a recognition algorithm.12. The method of claim 10, further comprising the steps of moving theunit to scan additional images; and combining the new image with aprevious images.
 13. The method of claim 10, further comprising thesteps of continuously scanning images from the camera.
 14. The method ofclaim 13, further comprising the steps of updating the position of theidentified objects in the database if these objects are moved.
 15. Amethod of allowing a user of a portable device to find an objectcomprising the steps of; searching the database for an absolute locationof the object; determining a current location of the user; determiningthe absolute position of the object; mapping a path from the currentlocation to the absolute position; and issuing directions for the userof the portable device to move along the path; thereby allowing the userto find the object.
 16. The method of claim 15, further comprising thesteps of moving the device to scan additional images; and combining thenew image with a previous images.
 17. The method of claim 15, whereinthe directions are displayed on a display screen of the portable device.18. The method of claim 15, wherein the directions are audible emittedfrom the portable device.
 19. The method of claim 15, wherein thecurrent position of the object is determined by a recognition algorithm.20. The method of claim 15, wherein the absolute position of the objectis determined by a recognition algorithm.